Method and system for displaying multi-viewpoint images and non-transitory computer readable storage medium thereof

ABSTRACT

A system for displaying multi-viewpoint images includes a terminal device having a communication component, a user interface (UI) component and a processing component. The communication component receives an image and a polygonal image model for establishing the relationship between each image and a corresponding viewing angle; one of the viewpoint angles is the current viewpoint angle. The UI component generates a UI for presenting the correspondence between the image of the current viewpoint angle and its viewpoint angle; the correspondence is used for selecting a target-viewpoint angle from the viewpoint angles. The processing component generates a switch instruction based on the target-viewpoint angle and allows UI to present an image corresponding to the target-viewpoint angle after UI presents an interval image being an image of an interval viewpoint angle between the current viewpoint angle and the target-viewpoint angle and is generated based on the image of the target-viewpoint angle.

RELATED APPLICATIONS

This application claims priority to Taiwan Application Serial Number102121560, filed Jun. 18, 2013, which is herein incorporated byreference.

TECHNICAL FIELD

The present disclosure relates to a display system, method andnon-transitory compute readable recording medium for displayingmulti-viewpoint images, and more particularly, relates to a displaysystem, method and non-transitory computer-readable recording medium fordisplaying multi-viewpoint images in a terminal device.

BACKGROUND

With the technological development goes toward providing personalizedservices, the interaction between digital contents and users are gettingmore and more vital. In essence, most currently-available interactivecontents are the aggregation of existing contents and interactionthrough social network, such as rating, voting, sharing, etc; and theyare not real interactive contents for the users to interact withthemselves. The display of media contents, on the other hand, has beenpursuing presence experiences that are more vivid or real. However, thepresentation of the media content provided by the media providerconsists of viewpoint angles that are subjectively screen or switch bythe film director or the program director during the broadcasting orrecording of the media content. It cannot provide a more flexible orpersonalized service to the viewers, and hence the viewers can onlypassively receive the display of single and limited scenes. Therefore,if it is feasible to provide the images that allow the viewers to selectviewpoint in multiple viewpoint angles, it will be possible to bring theviewers a more vivid experience that mimics the real world. Accordingly,in addition to the operations of playing, pausing, and forwarding, theviewers are allowed to freely select mages of other viewpoint angles, sothat they can track the characters or scenes of interest. In this way,the viewers can have the options of content selections.

U.S. Pat. No. 5,574,836 discloses a method of changing three-dimensionalimages according to the position of the user. The method uses a meansfor determining the position of a viewer and a graphics engine forswitching the graphics instructions into data for presenting in adisplay; and when the viewer changes his position, the method simulatesand presents three-dimensional images in different viewpoint angle.However, this method requires a great deal of calculation to process thesimulation of three-dimensional images, and the simulated images tend toloss their fidelity. Also, it can only process a single image source andcannot simulate a series of images, nor to be applied in images for livebroadcasting.

US patent application publication No. US 2010/0026788 discloses a methodfor generating free viewpoint images in three-dimensional movement. Themethod includes taking multi-viewpoint video images using a plurality ofcameras located on an identical plane that recording the object byhorizontally circumventing the object and a camera located above theobject, and then uses a processing procedure to generate virtualviewing-angle images that are parallel to the plane of the plurality ofcameras or of other angles. However, this method only allows the use ofa plurality of cameras to record one object at a time, and since thedata are huge and the images must undergo post-production processing,this method cannot be applied in images of large scenes or images forlive broadcasting. Also, this method does not provide operationalinterface that allows the user to select the viewpoint angle freely.

Further, U.S. Pat. No. 8,189,036 discloses method for combining abackground image modeling and a projection switching matrix to generatemulti-viewpoint images. The sender establishes a scene background modelthrough consecutive images, the receive receives the current image, theimage depth (depth Map), the background model and the projectionswitching matric, to convert the current image through the projectionswitching matric, which is then combined with the background image tosynthesize the virtual viewing-angle image. However, it failed toprovide an interactive operation method, and the data transmissionbetween the sender and the receiver does not include methods foradaptive content compression and transmission. Hence, the data are hugeand is not applicable in live broadcasting or common terminal devices.

In view of the foregoing, there exist problems and disadvantages in therelated art for further improvement; however, those skilled in the artsought vainly for a suitable solution. In view of the foregoing, thereis an urgent need in the related field regarding allowing users tofreely switch among different viewpoint angles while viewing the videoso as to obtain a more vivid presence experience, providing users withsimple and smooth experience of personalized interactive services.

SUMMARY

The following presents a simplified summary of the disclosure in orderto provide a basic understanding to the reader. This summary is not anextensive overview of the disclosure and it does not identifykey/critical components of the present invention or delineate the scopeof the present invention. Its sole purpose is to present some conceptsdisclosed herein in a simplified form as a prelude to the more detaileddescription that is presented later.

In view of the foregoing, one purpose of the present disclosure is toprovide system and method for displaying a multi-viewpoint image, sothat the user may freely select a viewpoint angle when he/she views theimage using a terminal device. In the system and method of the presentdisclosure, a polygonal image model is used to reduce the datathroughput of the multi-viewpoint images, and hence, the present systemand method are particularly suitable for use in the display of liveimages, and can be used in a common terminal device. Moreover, thesystem and method of the present disclosure uses a user interface sothat the user may conveniently operate and make selection, and switchthe viewpoint angle; further, by using the processing of a processingcomponent, when the user switches the viewpoint angle, a plurality ofinterval images are generated and presented to bridge the images beforeand after the switch, thereby improving the interactivity, the userexperience and fluency of the display of multi-viewpoint images. Ascould be appreciated, the multi-viewpoint image described throughout thepresent disclosure can be one or more still, static images, or one ormore video clips consisting of multiple moving, kinetic images.

According to one embodiment of the present disclosure, a display systemfor displaying multi-viewpoint images comprises a terminal device, andthe terminal device comprises a communication component, a userinterface component and a processing component. The communicationcomponent is configured to receive a plurality of images and a polygonalimage model, wherein the polygonal image model establishes arelationship between each image of the plurality of images and acorresponding viewpoint angle of each image, and one of the plurality ofviewpoint angles corresponding to the plurality of images is a currentviewpoint angle; the user interface component is configured to generatea user interface for presenting the images corresponding to the currentviewpoint angle, and a correspondence between the correspondingviewpoint angles for the plurality of images, wherein the correspondenceis provided for selecting a target-viewpoint angle from the viewpointangles; the processing component is configured to generate a switchinstruction based on the selected target-viewpoint angle, to allow theuser interface to display a plurality of interval images and thenpresent the image corresponding to the target-viewpoint angle, whereinthe plurality of interval images is according to the image correspondingto the target-viewpoint angle and the image corresponding to the currentviewpoint angle according to the polygonal image model and be generatedby a plurality of interval viewpoint angles between the currentviewpoint angle and the target-viewpoint angle during the process ofswitching the current viewpoint angle to the target-viewpoint angle.

According to another embodiment of the present disclosure, a displaymethod for displaying multi-viewpoint images comprises the steps of:receiving a plurality of images and a polygonal image model, wherein thepolygonal image model establishes a relationship between each image ofthe plurality of images and corresponding viewpoint angle thereof, andone of a plurality of viewpoint angles corresponding to the plurality ofimages is a current viewpoint angle; using a user interface forpresenting the images corresponding to the current viewpoint angle, anda correspondence between the corresponding viewpoint angles for theplurality of images, wherein the correspondence is provided forselecting a target-viewpoint angle from the viewpoint angles; and usinga processing component to generate a switch instruction based on theselected target-viewpoint angle, to allow the user interface to render aplurality of interval images, and then present the image correspondingto the target-viewpoint angle, wherein the plurality of interval imagesis based on the image corresponding to the target-viewpoint angle andthe image corresponding to the current viewpoint angle according to thepolygonal image model and be generated by a plurality of intervalviewpoint angles between the current viewpoint angle and thetarget-viewpoint angle during the process of switching the currentviewpoint angle to the target-viewpoint angle.

According to still another embodiment of the present disclosure, anon-transitory computer-readable recording medium has a computer programstored thereon and is configured to perform a method for displayingmulti-viewpoint images in a display system for displayingmulti-viewpoint images, wherein the method for displayingmulti-viewpoint images comprises the steps of: receiving a plurality ofimages and a polygonal image model, wherein the polygonal image modelestablishes a relationship between each image of the plurality of imagesand corresponding viewpoint angle thereof, and one of a plurality ofviewpoint angles corresponding to the plurality of images is a currentviewpoint angle; using a user interface for presenting the imagescorresponding to the current viewpoint angle, and a correspondencebetween the corresponding viewpoint angles for the plurality of images,wherein the correspondence is provided for selecting a target-viewpointangle from the viewpoint angles; and using a processing component togenerate a switch instruction based on the selected target-viewpointangle, to allow the user interface to render a plurality of intervalimages, and then present the image corresponding to the target-viewpointangle, wherein the plurality of interval images is based on the imagecorresponding to the target-viewpoint angle and the image correspondingto the current viewpoint angle according to the polygonal image modeland be generated by a plurality of interval viewpoint angles between thecurrent viewpoint angle and the target-viewpoint angle during theprocess of switching the current viewpoint angle to the target-viewpointangle.

Technical advantages generally attained, by the foregoing embodiments ofthe present invention, include: reducing the data throughput of themulti-viewpoint images, thereby suitable for use in the display of liveimages, and in a common terminal device. The user may freely select aviewpoint angle, and when switching the viewpoint angles, the conversionof video images is fluent so that the user, when viewing the video, canhave a vivid experience and have a feeling of presence. Moreover, asimple and easy-to-use user interface is provided so as to allow anintuitive interaction.

Many of the attendant features will be more readily appreciated, as thesame becomes better understood by reference to the following detaileddescription considered in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The present description will be better understood from the followingdetailed description read in light of the accompanying drawing, wherein:

FIG. 1 is a block diagram of a display system for displayingmulti-viewpoint images according to one embodiment of the presentdisclosure;

FIG. 2 is a schematic view of a user interface according to anotherembodiment of the present disclosure;

FIG. 3 is a block diagram of a display system for displayingmulti-viewpoint images according to another embodiment of the presentdisclosure;

FIG. 4 is a schematic view of a three-dimensional visual model accordingto yet another embodiment of the present disclosure;

FIG. 5 is a block diagram of a display system for displayingmulti-viewpoint images according to still another embodiment of thepresent disclosure; and

FIG. 6 is a flow chart of a display method for displayingmulti-viewpoint images according to one embodiment of the presentdisclosure.

DETAILED DESCRIPTION

In the following detailed description, for purposes of explanation,numerous specific details are set forth in order to attain a thoroughunderstanding of the disclosed embodiments. In accordance with commonpractice, the various described features/elements are not drawn to scalebut instead are drawn to best illustrate specific features/elementsrelevant to the present invention. Also, like reference numerals anddesignations in the various drawings are used to indicate likeelements/parts. Moreover, well-known structures and devices areschematically shown in order to simplify the drawing and to avoidunnecessary limitation to the claimed invention.

Referring to FIG. 1 which is a display system for displayingmulti-viewpoint images according to one embodiment of the presentdisclosure. As illustrated in FIG. 1, the display system 100 fordisplaying multi-viewpoint images mainly comprises a terminal device110. The terminal device 110 comprises a communication component 112, auser interface component 114 and a processing component 116. Theterminal device 110 may be a specific device composed of above-mentionedcomponents, or be an electronic device with general specificationintegrated with the above-mentioned components. The electronic devicewith general specification can be a smart phone, a tablet computer, anotebook or a personal computer.

The communication component 112 is configured to receive a plurality ofimages and a polygonal image model, wherein the polygonal image model isused to establish a relationship between each image of the plurality ofimages and corresponding viewpoint angle thereof, and one of theplurality of viewpoint angles corresponding to the plurality of imagesis a current viewpoint angle. In certain embodiments, the polygonalimage model can be a set of recording data that record the viewpointangle corresponding to each image, and one of the viewpoint angles isthe current viewpoint angle. In some other embodiments, the polygonalimage model can be built by calculating the relative relationship of theimages captured by a plurality of cameras according to the position ofthese cameras in a shooting scene, so as to generate the viewpoint anglecorresponding to each image and the polygonal image model. In apreferred embodiment, the plurality of images can be the images of thesame content but captured from different viewpoint angles, whereas thecurrent viewpoint angle is the viewpoint angle corresponding to thecurrently displayed image. Preferably, the number of the image equals tothe number of the viewpoint angle; however, the present invention is notlimited thereto.

The user interface component 114 is configured to generate a userinterface for presenting the images corresponding to the currentviewpoint angle, and a correspondence between the correspondingviewpoint angles for the plurality of images, wherein the correspondenceis provided for selecting a target-viewpoint angle from these viewpointangles. In certain embodiments, the user interface can be an userinterface of applications, and the display of the terminal device 110 isconfigured to display the image corresponding to the current viewpointangle and provide a graphic user interface. The graphic user interfacecan be used to label the viewing orientation of each viewpoint angle inthe plurality of viewpoint angles, and perform a special labeling to thecurrent viewpoint angle. It provides an operating function to allow theuser to select one of the viewpoint angles as the target-viewpointangle. In some other embodiments, the user interface can also be a touchscreen, an integration of the button of the terminal device 110 andother display, or an integration of other human-machine interface and aprojector, etc. It is configured to display the images, present aplurality of viewpoint angles, and provides an operating function toallow the user to select one of the viewpoint angles as thetarget-viewpoint angle.

The processing component 116 is configured to generate a switchinstruction based on the selected target-viewpoint angle, to allow theuser interface to display a plurality of interval images and thenpresent the image corresponding to the target-viewpoint angle, whereinthe plurality of interval images is according to the image correspondingto the target-viewpoint angle and the image corresponding to the currentviewpoint angle according to the polygonal image model and be generatedby a plurality of interval viewpoint angles between the currentviewpoint angle and the target-viewpoint angle during the process ofswitching the current viewpoint angle to the target-viewpoint angle. Inthis way, the user, when viewing the video, can have a more vividexperience that mimics the real world. In certain embodiments, theprocessing component 116 can be a specific hardware component thatintegrates the program code and a processor/processing chip, or can be acommercially available processor (or a processor disposed in theterminal device 110), that is capable of reading specific program codesand performed functions associated with the codes.

The specific method for generating the interval image is providedhereinbelow in details. In one embodiment, each of the intervalviewpoint angles has an interval image corresponding thereto Theprocessing component 116 processes the image corresponding to thecurrent viewpoint angle and the image corresponding to thetarget-viewpoint angle based on the angular degrees of each intervalviewpoint angle, current viewpoint angle and the target-viewpoint angle,so as to respectively generate the corresponding interval images. Forexample, after the user interface displays the current image andprovides the operating function to allow the user to select thetarget-viewpoint angle, the processing component 116 is ready to switchfrom the current image to the image corresponding to thetarget-viewpoint angle, and generates the interval images to bedisplayed during the switching process. In certain embodiments, theprocessing component 116 determines the angular degree and number of theinterval viewpoint angle between the current viewpoint angle and thetarget-viewpoint angle based on the angular degrees of the currentviewpoint angle and the target-viewpoint angle, in which each intervalviewpoint angle has an interval image corresponding thereto.Furthermore, there is a transition order among the current viewpointangle, the interval viewpoint angles and the target-viewpoint angle. Thetransition order is determined primarily based on the angular degrees ofthe current viewpoint angle, the interval viewpoint angles and thetarget-viewpoint angle. It also can be determined by the sequentialrelationship during the switching from the current viewpoint angle andthe interval viewpoint angles to the target-viewpoint angle in aclockwise or counterclockwise direction. Thereafter, the processingcomponent 116 generates the interval images according to each intervalviewpoint angle The user interface displays the image corresponding tothe current viewpoint angle first, then respectively displays theinterval images corresponding to each of the interval viewpoint anglesaccording to the transition order, and finally displays the imagecorresponding to the target-viewpoint angle. In this way, when the userchanges the viewpoint angle of the video, he/she may experience a morefluent switching process.

In another embodiment, when the angle between the current viewpointangle and the target-viewpoint angle is quite large, there may be oneinterval viewpoint angle equals to or is similar to one of the viewpointangles in the polygonal image model among the plurality of intervalviewpoint angles between the current viewpoint angle and thetarget-viewpoint angle. The said one interval viewpoint angle can beused as a middle viewpoint angle. In other words, during the process ofswitching from the previous viewpoint angle to the target-viewpointangle, it may pass through one viewpoint angle of the viewpoint anglescorresponding to these images in the polygonal image model, and thisviewpoint angle is considered as the middle viewpoint angle. The currentviewpoint angle, the middle viewpoint angle and the target-viewpointangle have a switch order, and the middle viewpoint angle is one of theinterval viewpoint angles. That is, the processing component 116 canfirst switch from the current viewpoint angle to the middle viewpointangle, and then switch from the middle viewpoint angle to thetarget-viewpoint angle. In this way, the interval viewpoint anglesbetween the current viewpoint angle and the target-viewpoint angle canbe respectively assigned as a plurality of first interval viewpointangles between the current viewpoint angle and the middle viewpointangle, and as a plurality of second interval viewpoint angles betweenthe middle viewpoint angle and the target-viewpoint angle, according tothe switch order, wherein each first interval viewpoint angle and eachsecond interval viewpoint angle respectively correspond to an intervalimage. The processing component 116 is further configured to find, basedon the polygonal image model, the image corresponding to the middleviewpoint angle and the image corresponding to the target-viewpointangle, and then further combine the image corresponding to the currentviewpoint angle and the image corresponding to the middle viewpointangle to generate the plurality of corresponding interval imagesaccording to each first interval viewpoint angle, the current viewpointangle and the middle viewpoint angle; and further combine the imagecorresponding to the middle viewpoint angle and the image correspondingto the target-viewpoint angle to generate the plurality of correspondinginterval images according to each second interval viewpoint angle, themiddle viewpoint angle and the target-viewpoint angle. When the userswitches the viewpoint angle, the terminal device 110 does not directlyswitch between different images corresponding to different viewpointangles; rather, it integratively processes the images of the currentviewpoint angle and the target-viewpoint angle (according to the timingsequence) to generate a plurality of interval images, and uses the userinterface for displaying, thereby mimicking the effect that the usertravels through the field step-by-step.

In practice, whether it makes the user fell smooth may dependent on thenumber of the interval viewpoint angles and the switching time forswitching in the interval switching process. Generally, human eye canperceive 16 frames per second (FPS). In one embodiment of the presentdisclosure, during the interval switching process, the processingcomponent 116 determines that the number of the interval viewpoint angleenables the display of at least 16 images per second, so as to achieve asmooth switching process. For example, when the switching time neededfor the interval switching process is 0.5 seconds, the processingcomponent 116 can determines in advance that 8 images are required, inwhich the first 7 images are the interval images, and the 8^(th) imageis the target image. Accordingly, the number of the interval viewpointangle is 7. Then it divides the angular difference between the currentviewpoint angle and the target-viewpoint angle into 8 equal parts. Thestarting point is the current viewpoint angle, the first to seventhangles are the interval viewpoint angles, and the 8^(th) angle is thetarget-viewpoint angle. Based on the above-mentioned data, 7 intervalimages corresponding thereto are generated thereby allowing a smoothinterval switching process.

In some other embodiments, when the user wants to view the viewpointangle and image thereof that does not belong to the viewpoint angles andimages established in the polygonal image model, the present disclosurefurther provides a virtual viewpoint angle and image correspondingthereto, so as to provide the effect that the user may feel like stayingin the field arbitrary. The user interface component 114 is further usedby the user to set a virtual viewpoint angle. The processing component116 is further configured to generate a virtual viewpoint angleinstruction according to the virtual viewpoint angle set by the user, soas to find, from the viewpoint angles, two adjacent viewpoint anglesrespectively in adjacent to the two sides of the virtual viewpoint angleand images corresponding thereto according to the virtual viewpointangle and the polygonal image model. It further processes the imagescorresponding to the two adjacent viewpoint angles to generate the imagecorresponding to the virtual viewpoint angle, according to the virtualviewpoint angle and the relative angular degree of the two adjacentviewpoint angles. For example, after setting the virtual viewpointangle, the processing component 116 can find, from the polygonal imagemodel, two viewpoint angles whose angular degrees that are closest tothe angular degree of the virtual viewpoint angle, and these twoviewpoint angles can be used as the adjacent viewpoint angles. Then, therelative angular degrees between the virtual viewpoint angle and the twoadjacent viewpoint angles can be calculated, and the corresponding imagecan be generated according to the images corresponding to the twoadjacent viewpoint angles.

As illustrated in FIG. 2, the user interface 200 may present image 210corresponding to current viewpoint angle 212 and the correspondencebetween each image and the viewpoint angles 222, 232, 242 correspondingthereto, and label the current viewpoint angle 212 from these viewpointangles, so as to facilitate the user's understanding to the position ofthe current viewpoint angle. In this way, the present disclosureprovides a simple and easy-to-use user interface 200 so as to providethe user an intuitive interaction measure.

To further discuss the overall infrastructure of the display system 100for displaying multi-viewpoint images, referring to FIG. 3. FIG. 3 is ablock diagram of a display system for displaying multi-viewpoint imagesaccording to another embodiment of the present disclosure. It should benoted that the content related to those described in the aboveembodiments will not be repeated hereinbelow.

As illustrated in the embodiment of FIG. 3, the user interface component114 is a touch screen. The touch screen comprises a display module 142and a touch-control module 144. The display module 142 is configured topresent the user interface and the correspondence between said viewpointangles, and display the image corresponding to the current viewpointangle, the interval images and the image corresponding to thetarget-viewpoint angle. The touch-control module 144 is configured toreceive the user's path of slide gesture and generate the switchinstruction according to the path of slide gesture, wherein the path ofslide gesture is formed by the user operating on the correspondingrelationship among the viewpoint angles rendered by the display module142 to select the target-viewpoint angle. In practice, the slide gestureand the instruction represented by the path corresponding to the slidegesture can be flexibly designed depending on actual need.

In the embodiment of FIG. 3, the images in the display system 100 fordisplaying multi-viewpoint images can be captured by using a pluralityof cameras 151, 152, 153, 154 to respectively record a shooting scene.The display system 100 for displaying multi-viewpoint images may furthercomprise an image modeling device 130. The image modeling device 130 isconfigured to calculate a relative relationship among the plurality ofimages recorded by the plurality of cameras 151, 152, 153, 154, andgenerate the viewpoint angle corresponding to each image and thepolygonal image model according to the positions of the plurality ofcameras 151, 152, 153, 154 in the shooting scene 150.

The image modeling device 130 is configured to establish athree-dimensional visual model 160 (illustrated in FIG. 4) according tosaid images and viewpoint angles corresponding thereto; thethree-dimensional visual model 160 comprises one or more planes 161,162, 163, 164, wherein the number of the plane is corresponding to thenumber of the image, and each plane is respectively corresponding toeach image so as to display the images, and the perpendicular vector ofeach plane is respectively corresponding to the viewpoint angle of eachimage.

In another embodiment of the present disclosure, the display system 100for displaying multi-viewpoint images may further comprise a server 120and an adaptive internet transmitting device 170 The server 120 cancommunicate with the terminal device 100 via internet. The server 120 isconfigured to store the plurality of images, and communicate with theterminal device 100 via internet to transmit the plurality of images tothe terminal device 100. The adaptive internet transmitting device 170is configured to adjust an amount of the plurality of images to betransmitted to the terminal device 100 according to the bandwidth of theinternet, wherein the terminal device 100 adjusts the polygonal imagemodel until it only comprises the viewpoint angle corresponding to eachimage of the plurality of images that are transmitted according to theplurality of images that are transmitted and the amount thereof. Byusing this adaptive internet transmission and video compression, thepresent embodiment can provide different video display modes dependingon different network conditions, so as to provide the user a betterinteractive experience with the video.

In another embodiment, the server 120 can also integrate the function(s)of the image modeling device 130. The server 120 is configured tocalculate a relative position among the plurality of images recorded bythe plurality of cameras 151, 152, 153, 154, according to the positionsof these cameras 151, 152, 153, 154 in the shooting scene 150, and togenerate the viewpoint angle corresponding to each image and thepolygonal image model.

Regarding another technical solution of the user interface component114, the readers attention is brought to FIG. 5. FIG. 5 is a blockdiagram of a display system for displaying multi-viewpoint imagesaccording to still another embodiment of the present disclosure. Itshould be noted that the content related to those described in the aboveembodiments will not be repeated hereinbelow.

As illustrated in FIG. 5, the terminal device 110 further comprises anorientation sensor 118. The orientation sensor 118 is configured tosense an orientation change of the terminal device 100 in athree-dimensional space, generate the switch instruction and transmitthe switch instruction to the processing component 116. The userinterface component 114 comprises a display 146 and at least one button148. The display 146 is configured to display the user interface, andthe at least one button 148 is used by the user to select thetarget-viewpoint angle from the viewpoint angles. In practice, theorientation sensor 118 can be an electronic compass, a gyroscope, andaccelerator or a combination thereof.

Referring to FIG. 6. FIG. 6 is a flow chart of a display method 300 fordisplaying multi-viewpoint images according to one embodiment of thepresent disclosure. The display method 300 for displayingmulti-viewpoint images can be implemented by the above-described displaysystem for displaying multi-viewpoint images. Alternatively, the displaymethod 300 could also be implemented as a computer program and stored ina computer-readable recording medium, so as to allow the computer, afterreading said recording medium, can perform the method with thecollaboration of other associated hardware components. Saidcomputer-readable recording medium can be a read-only memory, flashmemory, floppy disk, hard drive, compact disk, USB drive, magnetic tape,a database accessible via the internet, or any other computer-readablerecording medium that can be easily contemplated by persons havingordinary skill in the art and capable of performing the samefunction(s).

As illustrated in FIG. 6, the display method 300 for displayingmulti-viewpoint images comprises steps 302 to 306 (it should be notedthat for the steps described in the present embodiment, the sequence inwhich these steps is performed, unless explicitly stated otherwise, canbe altered depending on actual needs. In certain cases, all or some ofthese steps can be performed concurrently); as to the hardware devicesrequired for the implementation of these steps, they have beenspecifically disclosed in the above-mentioned embodiments, and hencewill not be repeated hereinbelow.

In step 302, a communication component is used to receive a plurality ofimages and polygonal image model. The polygonal image model establishesa relationship between each image of the plurality of images andcorresponding viewpoint angle thereof, and one of a plurality ofviewpoint angles corresponding to the plurality of images is a currentviewpoint angle.

In step 304, a user interface component is used to generate a userinterface. Also, the user interface is used to present the imagecorresponding to the current viewpoint angle and a correspondencebetween each of the corresponding viewpoint angles for the plurality ofimages. The correspondence is provided for selecting a target-viewpointangle from the viewpoint angles.

In step 306, a processing component is used to generate a switchinstruction based on the selected target-viewpoint angle, to allow theuser interface to display a plurality of interval images. The processingcomponent is then used to present the image corresponding to thetarget-viewpoint angle, wherein the interval images is based on theimage corresponding to the target-viewpoint angle and the imagecorresponding to the current viewpoint angle according to the polygonalimage model and be generated by a plurality of interval viewpoint anglesbetween the current viewpoint angle and the target-viewpoint angle,during the process of switching the current viewpoint angle to thetarget-viewpoint angle. In this way, the user, when viewing the video,can have a more vivid experience that mimics the real world. It shouldbe noted that the content related to those described in the aboveembodiments, such as, the polygonal image model, the user interface andthe processing component, will not be repeated in the embodimentsrelated to the present method.

The specific method for generating the interval image is described indetailed hereinbelow. In one embodiment, the display method 300 fordisplaying multi-viewpoint images may further comprise: using theprocessing component to process the image corresponding to the currentviewpoint angle and the image according to the target-viewpoint anglebased on the angular degrees of each interval viewpoint angle, currentviewpoint angle and the target-viewpoint angle, and to respectivelygenerate the plurality of corresponding interval images based on thecurrent viewpoint angle and the target-viewpoint angle. For instance,after the user interface displays the current image and provides theoperating function to allow the user to select the target-viewpointangle, the processing component prepares to switch from the currentimage to the image corresponding to the target-viewpoint angle, and togenerate the interval image(s) to be displayed during the switchingprocess. In certain embodiments, the processing component determines theangular degree and number of the interval viewpoint angle between thecurrent viewpoint angle and the target-viewpoint angle, based on theangular degrees of the current viewpoint angle and the target-viewpointangle. Each interval viewpoint angle has an interval image correspondingthereto. Moreover, the step 306 may further comprise: using antransition order among the current viewpoint angle, the intervalviewpoint angles and the target-viewpoint angle to determine thesequential relationship, during the switching from the current viewpointangle and the interval viewpoint angles to the target-viewpoint angle.The transition order is determined primarily based on the angulardegrees of the current viewpoint angle, the interval viewpoint anglesand the target-viewpoint angle; for example, in a clockwise orcounterclockwise direction. Then, the processing component generates theinterval images according to each interval viewpoint angle, so that theuser interface first displays the image corresponding to the currentviewpoint angle. Next the user interface respectively displays theinterval images corresponding to each of the interval viewpoint anglesaccording to the transition order, and then displays the imagecorresponding to the target-viewpoint angle. In this way, when the userchanges the viewpoint angle of the video, he/she may experience a morefluent switching process.

In another embodiment of the present disclosure, when the angulardifference between the current viewpoint angle and the target-viewpointangle is quite large, among the plurality of interval viewpoint anglesbetween the current viewpoint angle and the target-viewpoint angle,there may be one interval viewpoint angle equals to or is similar to oneof the viewpoint angles in the polygonal image model. Said one intervalviewpoint angle can be used as a middle viewpoint angle. In other words,during the process of switching from the previous viewpoint angle to thetarget-viewpoint angle, it may pass through one viewpoint angle of theviewpoint angles corresponding to these images in the polygonal imagemodel, and this viewpoint angle is considered as the middle viewpointangle. The display method 300 for displaying multi-viewpoint images mayfurther comprises: using the viewpoint angles corresponding to theplurality of images that comprise a middle viewpoint angle, the currentviewpoint angle, the middle viewpoint angle and the target-viewpointangle have a switch order, and the middle viewpoint angle is one of theplurality of interval viewpoint angles. That is, the processingcomponent 116 can first switch from the current viewpoint angle to themiddle viewpoint angle, and then switch from the middle viewpoint angleto the target-viewpoint angle. In this way, the interval viewpointangles between the current viewpoint angle and the target-viewpointangle can be respectively assigned as a plurality of first intervalviewpoint angles between the current viewpoint angle and the middleviewpoint angle, and as a plurality of second interval viewpoint anglesbetween the middle viewpoint angle and the target-viewpoint angle,according to the switch order. Each first interval viewpoint angle andeach second interval viewpoint angle respectively correspond to aninterval image. In this embodiment, the processing component furtherfinds, based on the polygonal image model, the image corresponding tothe middle viewpoint angle and the image corresponding to thetarget-viewpoint angle, and then further combines the imagecorresponding to the current viewpoint angle and the image correspondingto the middle viewpoint angle to generate the plurality of correspondinginterval images according to each first interval viewpoint angle, thecurrent viewpoint angle and the middle viewpoint angle. The processingcomponent further combines the image corresponding to the middleviewpoint angle and the image corresponding to the target-viewpointangle to generate the plurality of corresponding interval imagesaccording to each second interval viewpoint angle, the middle viewpointangle and the target-viewpoint angle. When the user switches theviewpoint angle, the present method does not directly switch betweendifferent images corresponding to different viewpoint angles. Rather,the user switches integratively processes the images of the currentviewpoint angle and the target-viewpoint angle (according to the timingsequence) to generate a plurality of interval images, and uses the userinterface for displaying, thereby mimicking the effect that the usertravels through the field step-by-step.

In some other embodiments, when the user wants to view the viewpointangle and image thereof that does not belong to the viewpoint angles andimages established in the polygonal image model, the present disclosurefurther provides a virtual viewpoint angle and image correspondingthereto, so as to provide the effect that the user may feel like stayingin the field arbitrary. The display method 300 for displayingmulti-viewpoint images may further comprising: using the user interfacecomponent to set a virtual viewpoint angl. The processing component isfurther configured to generate a virtual viewpoint angle instructionaccording to the virtual viewpoint angle, so as to find, from theviewpoint angles, two adjacent viewpoint angles respectively in adjacentto the two sides of the virtual viewpoint angle and images correspondingthereto according to the virtual viewpoint angle and the polygonal imagemodel. The processing component is further configured to process theimages corresponding to the two adjacent viewpoint angles, according tothe virtual viewpoint angle and the relative angular degree of the twoadjacent viewpoint angles, so as to generate the image corresponding tothe virtual viewpoint angle which is used as the main display image. Forexample, after setting the virtual viewpoint angle, the present methodcan find, from the polygonal image model, two viewpoint angles whoseangular degrees that are closest to the angular degree of the virtualviewpoint angle. Then, the relative angular degrees between the virtualviewpoint angle and the two adjacent viewpoint angles can be calculated,and the corresponding image can be generated and displayed.

Regarding the detailed means for the implementation of the userinterface, in one embodiment of the present disclosure, the displaymethod 300 for displaying multi-viewpoint images may comprise:presenting the user interface and the correspondence between saidviewpoint angles, and displaying the image corresponding to the currentviewpoint angle, the interval images and the image corresponding to thetarget-viewpoint angle; receiving the user's path of slide gesture andgenerating the switch instruction according to the path of slidegesture. The path of slide gesture is formed by the user operating onthe corresponding relationship among the viewpoint angles rendered bythe display module to select the target-viewpoint angle. In practice,the slide gesture and the instruction represented by the pathcorresponding to the slide gesture can be flexibly designed depending onactual need.

In one embodiment of the present disclosure, the display method 300 fordisplaying multi-viewpoint images may comprise the steps of: using aserver to store the plurality of images, and communicating with theterminal device via internet to transmit the images to the terminaldevice. The terminal device comprises the processing component and theuser interface component for generating the user interface. Next, thedisplay method 300 comprises adjusting an amount of the images to betransmitted to the terminal device according to the bandwidth of theInternet, until the polygonal image model only comprises the viewpointangle corresponding to each image of the transmitted images. By usingthis adaptive Internet transmission and video compression, the presentembodiment can provide different video display modes depending ondifferent network conditions, so as to provide the user a betterinteractive experience with the video.

In view of the foregoing, the present disclosure a more vividvideo-viewing experience that mimics the real world. The user can alterthe viewpoint angle of the video, and experience a more fluent videoswitch. Also, the present disclosure provides a simple andeasy-to-operate user interface, and adaptive internet transmission andvideo compression, thereby providing a means to provide different videodisplay modes depending on the network condition, so that user can havea better interactive experience with the video.

Although various embodiments of the invention have been described abovewith a certain degree of particularity, or with reference to one or moreindividual embodiments, they are not limiting to the scope of thepresent disclosure. Those with ordinary skill in the art could makenumerous alterations to the disclosed embodiments without departing fromthe spirit or scope of this invention. Accordingly, the protection scopeof the present disclosure shall be defined by the accompany claims.

What is claimed is:
 1. A display system for displaying multi-viewpointimages, comprising: a terminal device, comprising: a communicationcomponent, configured to receive a plurality of images and a polygonalimage model, wherein the polygonal image model establishes arelationship between each image of the plurality of images andcorresponding viewpoint angle thereof, and one of the plurality ofviewpoint angles corresponding to the plurality of images is a currentviewpoint angle; a user interface component, configured to generate auser interface for presenting the images corresponding to the currentviewpoint angle, and a correspondence between the correspondingviewpoint angles for the plurality of images, wherein the correspondenceis provided for selecting a target-viewpoint angle from the viewpointangles; and a processing component, configured to generate a switchinstruction based on the selected target-viewpoint angle, to allow theuser interface to present a plurality of interval images and thenpresent the image corresponding to the target-viewpoint angle, whereinthe plurality of interval images is according to the image correspondingto the target-viewpoint angle and the image corresponding to the currentviewpoint angle according to the polygonal image model and be generatedby a plurality of interval viewpoint angles between the currentviewpoint angle and the target-viewpoint angle during the process ofswitching the current viewpoint angle to the target-viewpoint angle;wherein the viewpoint angles corresponding to the plurality of imagescomprise a middle viewpoint angle; the current viewpoint angle, themiddle viewpoint angle and the target-viewpoint angle have a switchorder, and the middle viewpoint angle is one of the plurality ofinterval viewpoint angles; the plurality of interval viewpoint anglesbetween the current viewpoint angle and the target-viewpoint angle arerespectively assigned as a plurality of first interval viewpoint anglesbetween the current viewpoint angle and the middle viewpoint angle and aplurality of second interval viewpoint angles between the middleviewpoint angle and the target-viewpoint angle according to the switchorder, wherein each first interval viewpoint angle and each secondinterval viewpoint angle respectively correspond to an interval image;and the processing component is further configured to find, based on thepolygonal image model, the image corresponding to the middle viewpointangle and the image corresponding to the target-viewpoint angle, furthercombine the image corresponding to the current viewpoint angle and theimage corresponding to the middle viewpoint angle to generate theplurality of corresponding interval images according to each firstinterval viewpoint angle and a relative angular degree between thecurrent viewpoint angle and the middle viewpoint angle, and furthercombine the image corresponding to the middle viewpoint angle and theimage corresponding to the target-viewpoint angle to generate theplurality of corresponding interval images according to each secondinterval viewpoint angle.
 2. The display system for displayingmulti-viewpoint images according to claim 1, wherein each intervalviewpoint angle respectively corresponds to one of the plurality ofinterval images; and the processing component respectively generates theplurality of corresponding interval images according to the imagecorresponding to the current viewpoint angle and the image correspondingto the current viewpoint angle, based on the angular degrees of eachinterval viewpoint angle, the current viewpoint angle and thetarget-viewpoint angle.
 3. The display system for displayingmulti-viewpoint images according to claim 1, wherein the user interfacecomponent is further used for setting a virtual viewpoint angle; and theprocessing component is further configured to generate a virtualviewpoint angle instruction according to the set virtual viewpointangle, to find, from the viewpoint angles, two adjacent viewpoint anglesrespectively in adjacent to the two sides of the virtual viewpoint angleand images corresponding thereto according to the virtual viewpointangle and the polygonal image model, and further to process the imagescorresponding to the two adjacent viewpoint angles to generate the imagecorresponding to the virtual viewpoint angle, according to the virtualviewpoint angle and the relative angular degree of the two adjacentviewpoint angles.
 4. The display system for displaying multi-viewpointimages according to claim 1, wherein the user interface component is atouch screen, and the touch screen comprising: a display module,configured to present the user interface and a correspondingrelationship among the viewpoint angles, and display the imagecorresponding to the current viewpoint angle, the plurality of intervalimages and the image corresponding to the target-viewpoint angle; and atouch-control module, configured to receive a path of slide gesture, andgenerate the switch instruction according to the path of slide gesture,wherein the path of slide gesture is formed by the user operating on thecorresponding relationship among the viewpoint angles presented by thedisplay module to select the target-viewpoint angle.
 5. The displaysystem for displaying multi-viewpoint images according to claim 1,wherein the terminal device further comprises: an orientation sensor,configured to sense an orientation change of the terminal device in athree-dimensional space, and generate the switch instruction andtransmit the switch instruction to the processing component, wherein theuser interface component is a display and at least one button, thedisplay being configured to display the user interface, and the at leastone button being used by the user to select the target-viewpoint anglefrom the viewpoint angles.
 6. The display system for displayingmulti-viewpoint images according to claim 1, wherein the plurality ofimages is obtained by using a plurality of cameras to respectivelyrecord a shooting scene, and the display system further comprises: animage modeling device, configured to calculate a relative relationshipamong the plurality of images recorded by the plurality of cameras, andgenerate the viewpoint angle corresponding to each image and thepolygonal image model according to the positions of the plurality ofcameras in the shooting scene.
 7. The display system for displayingmulti-viewpoint images according to claim 1, further comprising: aserver, configured to store the plurality of images, and communicatewith the terminal device via internet to transmit the plurality ofimages to the terminal device; and an adaptive internet transmittingdevice, configured to adjust an amount of the plurality of images to betransmitted to the terminal device according to the bandwidth of theinternet, wherein the terminal device adjusts the polygonal image modeluntil it only comprises the viewpoint angle corresponding to each imageof the plurality of images that are transmitted according to theplurality of images that are transmitted and the amount thereof.
 8. Amethod for displaying multi-viewpoint images, comprising: receiving aplurality of images and a polygonal image model, wherein the polygonalimage model establishes a relationship between each image of theplurality of images and corresponding viewpoint angle thereof, and oneof a plurality of viewpoint angles corresponding to the plurality ofimages is a current viewpoint angle; using a user interface forpresenting the images corresponding to the current viewpoint angle, anda correspondence between the corresponding viewpoint angles for theplurality of images, wherein the correspondence is provided forselecting a target-viewpoint angle from the viewpoint angles; using aprocessing component to generate a switch instruction based on theselected target-viewpoint angle, to allow the user interface to presenta plurality of interval images, and then present the image correspondingto the target-viewpoint angle, wherein the plurality of interval imagesis based on the image corresponding to the target-viewpoint angle andthe image corresponding to the current viewpoint angle according to thepolygonal image model and be generated by a plurality of intervalviewpoint angles between the current viewpoint angle and thetarget-viewpoint angle during the process of switching the currentviewpoint angle to the target-viewpoint angle; and using the viewpointangles corresponding to the plurality of images that comprise a middleviewpoint angle, the current viewpoint angle, the middle viewpoint angleand the target-viewpoint angle have a switch order, and the middleviewpoint angle is one of the plurality of interval viewpoint angles;the plurality of interval viewpoint angles between the current viewpointangle and the target-viewpoint angle being respectively assigned as aplurality of first interval viewpoint angles between the currentviewpoint angle and the middle viewpoint angle, and as a plurality ofsecond interval viewpoint angles between the middle viewpoint angle andthe target-viewpoint angle according to the switch order; wherein eachfirst interval viewpoint angle and each second interval viewpoint anglerespectively correspond to an interval image; and the processingcomponent is further configured to find, based on the polygonal imagemodel, the image corresponding to the middle viewpoint angle and theimage corresponding to the target-viewpoint angle, further combine theimage corresponding to the current viewpoint angle and the imagecorresponding to the middle viewpoint angle to generate the plurality ofcorresponding interval images, according to each first intervalviewpoint angle and a relative angular degree between the currentviewpoint angle and the middle viewpoint angle, and further combine theimage corresponding to the middle viewpoint angle and the imagecorresponding to the target-viewpoint angle to generate the plurality ofcorresponding interval images, according to each second intervalviewpoint angle.
 9. The method for displaying multi-viewpoint imagesaccording to claim 8, further comprising: using the processing componentto process the image corresponding to the current viewpoint angle andthe image corresponding to the current viewpoint angle, based on theangular degrees of each interval viewpoint angle, and to respectivelygenerate the plurality of corresponding interval images based on thecurrent viewpoint angle and the target-viewpoint angle.
 10. The methodfor displaying multi-viewpoint images according to claim 8, furthercomprising: using the user interface component to set a virtualviewpoint angle, the processing component further generating a virtualviewpoint angle instruction according to the set virtual viewpointangle, to find, from the viewpoint angles, two adjacent viewpoint anglesrespectively in adjacent to the two sides of the virtual viewpoint angleand images corresponding thereto, according to the virtual viewpointangle and the polygonal image model, and further to process the imagescorresponding to the two adjacent viewpoint angles to generate the imagecorresponding to the virtual viewpoint angle, according to the virtualviewpoint angle and the relative angular degree of the two adjacentviewpoint angles.
 11. The method for displaying multi-viewpoint imagesaccording to claim 8, further comprising: presenting the user interfaceand a corresponding relationship among the viewpoint angles, anddisplaying the image corresponding to the current viewpoint angle, theplurality of interval images and the image corresponding to thetarget-viewpoint angle; and receiving a path of slide gesture, andgenerating the switch instruction according to the path of slidegesture, wherein the path of slide gesture is formed by the useroperating on the corresponding relationship among the viewpoint anglespresented by the display module to select the target-viewpoint angle.12. The method for displaying multi-viewpoint images according to claim8, further comprising: using an orientation sensor to sense anorientation change in a three-dimensional space, and generate the switchinstruction and transmit the switch instruction to the processingcomponent, and displaying the user interface in a display, and using abutton such that the user uses the button to select the target-viewpointangle from the viewpoint angles.
 13. The method for displayingmulti-viewpoint images according to claim 8, further comprising:calculating the relative relationship between the plurality of imagesrecorded by a plurality of cameras, and generate the viewpoint anglecorresponding to each image and the polygonal image model according tothe positions of the plurality of cameras in the shooting scene.
 14. Themethod for displaying multi-viewpoint images according to claim 8,further comprising: using a server to store the plurality of images, andcommunicate with the terminal device via internet to transmit theplurality of images to the terminal device; wherein the terminal devicecomprises the processing component and a user interface component togenerate the user interface; and adjusting an amount of the plurality ofimages to be transmitted to the terminal device, according to thebandwidth of the internet, wherein the terminal device adjusts thepolygonal image model until it only comprises the viewpoint anglecorresponding to each image of the plurality of images that aretransmitted according to the plurality of images that are transmittedand the amount thereof.
 15. A non-transitory computer-readable recordingmedium, having a computer program stored thereon, configured to performa method for displaying multi-viewpoint images in a display system fordisplaying multi-viewpoint images, the method for displayingmulti-viewpoint images comprising: receiving a plurality of images and apolygonal image model, wherein the polygonal image model establishes therelationship between each image of the plurality of images andcorresponding viewpoint angle thereof, and one of a plurality ofviewpoint angles corresponding to the plurality of images is a currentviewpoint angle; using a user interface for presenting the imagescorresponding to the current viewpoint angle, and a correspondencebetween the plurality of images and the corresponding viewpoint anglethereof, wherein the correspondence is used for selecting atarget-viewpoint angle from the viewpoint angles; using a processingcomponent to generate a switch instruction based on the selectedtarget-viewpoint angle, to allow the user interface to present aplurality of interval images, and then present the image correspondingto the target-viewpoint angle, wherein the plurality of interval imagesis based on the image corresponding to the target-viewpoint angle andthe image corresponding to the current viewpoint angle according to thepolygonal image model and be generated by a plurality of intervalviewpoint angles between the current viewpoint angle and thetarget-viewpoint angle during the process of switching the currentviewpoint angle to the target-viewpoint angle; and using the viewpointangles corresponding to the plurality of images that comprise a middleviewpoint angle, the current viewpoint angle, the middle viewpoint angleand the target-viewpoint angle have a switch order, and the middleviewpoint angle is one of the plurality of interval viewpoint angles;the plurality of interval viewpoint angles between the current viewpointangle and the target-viewpoint angle being respectively assigned as aplurality of first interval viewpoint angles between the currentviewpoint angle and the middle viewpoint angle, and as a plurality ofsecond interval viewpoint angles between the middle viewpoint angle andthe target-viewpoint angle according to the switch order; wherein eachfirst interval viewpoint angle and each second interval viewpoint anglerespectively correspond to an interval image; and the processingcomponent is further configured to find, based on the polygonal imagemodel, the image corresponding to the middle viewpoint angle and theimage corresponding to the target-viewpoint angle, further combine theimage corresponding to the current viewpoint angle and the imagecorresponding to the middle viewpoint angle to generate the plurality ofcorresponding interval images, according to each first intervalviewpoint angle and a relative angular degree between the currentviewpoint angle and the middle viewpoint angle, and further combine theimage corresponding to the middle viewpoint angle and the imagecorresponding to the target-viewpoint angle to generate the plurality ofcorresponding interval images, according to each second intervalviewpoint angle.